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ABSTRACT 


Millions of women worldwide are affected by breast cancer, which is a serious global health issue. The likelihood of 
successful therapy and the prognosis both greatly benefit from early identification. The most popular screening method 
for breast cancer, mammography, produces precise biological images that can help with the early detection of 
malignancies. However, it is still difficult to correctly interpret mammography pictures, which frequently results in 
false positives or negatives. This study attempts to create a biological mammogram based deep learning system for 
breast cancer diagnosis. Convolutional neural networks (CNNs) are used to automatically identify and analyse 
mammogram pictures in the proposed system, enabling radiologists to make quicker and more accurate diagnoses. To 
ensure the best performance during the training phase, these photos underwent preprocessing to reduce noise and 
enhance characteristics. The deep learning model used is a cutting-edge CNN architecture that was pretrained on a 
sizable dataset to fully utilise its learned representations. The deep learning model underwent thorough training, 
validation, and fine-tuning procedures to ensure robustness and generalizability. A variety of data augmentation 
methods, including rotation, scaling, and flipping, was used to enlarge and diversify the dataset during training. To 
further increase the model's accuracy, transfer learning was used to utilize knowledge from other similar tasks. Using a 
variety of criteria, such as sensitivity, specificity, accuracy, and F1 score, and the performance of the created breast 
cancer detection system was carefully assessed. The results showed a substantial increase in accuracy when compared 
to traditional mammography analysis methods. The method demonstrated impressive specificity in reducing false 
positives and sensitivity in identifying actual positive situations. 
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1. INTRODUCTION image analysis to solve these issues. Convolutional 


One of the most common and dangerous 
cancers affecting women globally is breast cancer. 
Being responsible for a considerable portion of 
cancer-related deaths each year, it is a significant 
public health concern. In order to lower death rates 
and increase the likelihood of successful therapy, 
early identification is essential. As a widely used 
screening method, mammography has been crucial 
in detecting breast cancer in its early stages, 
allowing for prompt interventions and better patient 
outcomes. However, mammography image 
interpretation is a difficult, time-consuming process 
that frequently involves human error and _ inter- 
observer variability [1 ]-[2]. 


Recent developments in deep learning, a 
branch of artificial intelligence, have showed 
considerable promise in transforming medical 


neural networks (CNNs), in particular, have shown 
amazing ability in feature extraction and pattern 
recognition, making them suitable for challenging 
picture classification applications. Researchers have 
investigated the use of deep learning methods for 
breast cancer screening utilizing biomedical 
mammography images by leveraging this potential 


[3]. 


Creating an accurate and effective deep 
learning-based system for automated breast cancer 
diagnosis is the main goal of this research effort. 
The initiative intends to increase the overall 
efficacy of breast cancer screening programs by 
assisting radiologists in making more accurate and 
timely diagnosis by utilizing the capabilities of deep 
learning. This study is significant because it has the 
potential to address a number of pressing issues 
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related to breast cancer screening. First off, the 
suggested method can greatly lessen the subjectivity 
and variability linked to human interpretation by 
automating the detection process. This will result in 
more reliable results that are consistent and 
reproducible, enhancing the overall validity of 
mammography as a screening method. The model 
can learn a deep understanding of the many 
manifestations of breast cancer by being trained on 
a large database of annotated mammography 
pictures. The model will be able to distinguish 
between benign and malignant observations 
because to this extensive knowledge [6]-[8]. 


The research will also investigate the 
application of transfer learning, a method that 
makes use of pre-trained models on sizable datasets 
from comparable jobs. This method enables the 
deep learning model to take advantage of the 
information learned during its earlier training, 
potentially lowering the volume of labelled data 
needed for training and enhancing generalization to 
new and unseen data. 


2. LITERATURE REVIEW 

Early identification of breast cancer is 
essential for enhancing patient outcomes and 
lowering mortality rates. Breast cancer is a 
significant worldwide health issue. The most 
popular screening tool for breast cancer is 
mammography, which produces comprehensive 
biological images that radiologists may examine to 
find any potential malignancies. The complexity 
and subtlety of breast tumors make it difficult to 
interpret mammography pictures, which can result 
in both false positives and false negatives. Deep 
learning techniques, in particular convolutional 
neural networks (CNNs), have recently become 
effective tools for image analysis, holding the 
promise of revolutionizing the identification of 
breast cancer from mammography pictures. 


Numerous studies have investigated the 
use of deep learning in mammogram-based breast 
cancer screening. A deep learning model utilizing a 
sizable dataset of mammograms, and_ the 
researchers saw promising results in terms of 
sensitivity and specificity. The study proved that it 
is possible to use CNNs to help radiologists find 
breast cancer early. In this field, transfer learning 
has also received a lot of attention. CNNs that had 
been trained on massive image datasets to illustrate 
the efficacy of transfer learning. The model 
improved performance with less training data, 
overcoming the problem of data scarcity in medical 
imaging, by fine-tuning the pre-trained CNNs using 
mammography images [4]. Deep learning has also 


been used to investigate the possibility of ensemble 
approaches for the identification of breast cancer. In 
order to achieve more accuracy and robustness than 
individual models, presented an ensemble strategy 
that merged multiple CNN models. Ensembling has 
shown that it can improve the accuracy of breast 
cancer detection systems and lower the likelihood 
of misdiagnosis [7]. 


Interpretability is a fundamental 
component of deep learning models for the study of 
medical images. Researchers have made progress in 
illuminating CNN model decisions for breast cancer 
detection. For instance, Grad-CAM, a method that 
highlights the mammography regions that the model 
considered most important. Radiologists can gain 
useful insights from interpretability techniques like 
Grad-CAM, which boosts their confidence in AI- 
assisted diagnosis. 


Despite the advances, there are still certain 
difficulties when using deep learning to diagnose 
breast cancer. The security and privacy of data are 
of utmost importance when working with medical 
photographs. Deep learning algorithms that protect 
privacy have been developed, guaranteeing patient 
confidentiality and facilitating joint research [9]. 


Furthermore, it is essential to generalize 
models to other populations and mammography 
systems. When using deep learning models in actual 
clinical settings, emphasized the significance of 
domain adaptation to reduce performance 
inconsistencies [10]-[13]. 


The literature on biomedical 
mammography images-based deep learning for 
breast cancer diagnosis shows _ substantial 
advancements and potential. The accuracy and 
effectiveness of breast cancer screening have the 
potential to be improved via CNNs and transfer 
learning. Model reliability is increased through 
assembling approaches, while model judgments are 
made more understandable by interpretability 
techniques. However, issues like data privacy, 
model generalization, and domain adaptation need 
more investigation. 


3. NEWLY PROPOSED SYSTEM 

A hybrid architecture model for breast 
cancer diagnosis using biomedical mammography 
images combines many deep learning methods or 
models to improve the precision and efficiency of 
the detection system. The hybrid architecture 
intends to address the difficulties in mammography 
analysis and enhance the overall efficiency of breast 
cancer detection by integrating the capabilities of 
various models. 
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Each component of the hybrid architecture 
may have a different function in the detecting 
process. For example, the model might include 
recurrent neural networks (RNNs) and 
convolutional neural networks (CNNs). The CNNs 
are excellent in deriving high-level representations 
from mammography pictures and extracting image 
features. They are capable of accurately capturing 
complex patterns and minute characteristics that are 
symptomatic of breast cancer. 


RNNs, on the other hand, can handle 
sequential data, such mammography sequences or 
time-series data. RNNs may be used in the context 
of breast cancer diagnosis to examine temporal 
patterns or changes in mammography images 
obtained across a number of screenings, assisting in 
the detection of developing abnormalities that might 
be suggestive of cancer development. Additionally, 
the hybrid architecture could contain transfer 
learning. The mammography dataset can be used to 
fine-tune pre-trained CNNs from large-scale picture 
datasets, such as ImageNet, by utilizing their learnt 
representations to speed convergence and improve 
the performance of the breast cancer detection 
model. Transfer learning is particularly useful in 
situations where there is a shortage of labelled 
training data. 


Ensemble methods, in which many deep 
learning models are integrated to create collective 
predictions, may potentially be a part of the hybrid 
architecture. The CNN-based architecture may be 
modified for each unique model, which would be 
trained with various hyperparameters or 
initializations. The danger of false positives or 
negatives in the breast cancer detection process can 
be decreased by the ensemble of models' ability to 
produce more reliable predictions by averaging or 
voting on their individual outputs. 


The hybrid architecture might also 
incorporate interpretability techniques to help 
radiologists understand the model's choices and 
gain useful information. Grad-CAM or attention 
mechanisms, for instance, may highlight the areas 
of the mammogram that the model concentrates on 
when generating a cancer prediction. This would 
assist to increase user confidence in the system's 
judgements and speed up the clinical validation 
process. The hybrid model intends to increase 
breast cancer detection's accuracy, dependability, 
and interpretability, which will ultimately result in 
an earlier diagnosis, more efficient therapies, and 
better patient outcomes. 


4. METHODOLOGY 

A unique approach that combines the 
phases of patch recognition and whole image 
classification into a single procedure is offered to 
overcome the difficulties associated with 
classifying huge complex images. This method, 
shown in, offers improved efficiency and 
performance in jobs like breast cancer screening 
using biomedical mammography images by 
eliminating the requirement for individual tuning of 
each stage. 


4.1 Combining Patch and Whole Image 
Classification 
Combining patch and whole image 


classification in a unified approach. A novel 
approach is presented, assuming an input patch X € 
RP“ and a patch classifier function f yielding f(x) € 
R°, where c denotes the number of classes (for 
example, benign calcification, malignant 
calcification, etc.). A heatmap-based whole image 
classifier is produced by applying f to the entire 
image M € R™*. A grid of probabilistic outputs 
representing the presence of each class in various 
areas of the image makes up the output f(M) € 
Ruvxc | 


4.2 Image-level classification, add top layers 

Using additional layers, the heatmap f(M) 
is further processed. The output of the patch 
classifier is added to a convolutional layer to 
broaden the receptive field. For instance, a patch 
classifier with a 224x224 receptive field and a 32 
stride layer appended expands the receptive field to 
228x228. These top layers extract detailed features 
that essentially scan the entire image for signs of 
cancer. The final complete image classification 
function is defined as h(M) = g(f(M)) € R4, where d 
denotes the classes for the whole image (for 
example, malignant, nonmalignant). These top 
layers are denoted as function g. 


4.3. End-to-End Training & Transfer Learning 

This architecture offers two significant 
advantages over traditional two-step methods. 
Firstly, the entire network can be jointly trained, 
preventing sub-optimal answers from every phase. 
Secondly, the trained network can be effectively 
transferred to different datasets without requiring 
ROI annotations. This is particularly beneficial in 
medical imaging where annotated datasets are 
limited and pricey. For instance, the big 
mammography database DDSM can be employed 
to develop a patch classifier, and then fine-tuned on 
additional databases using simply image-level 
labels. 
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4.4 CNN Architectures 

The proposed methodology employs two 
well-known CNN structures: the VGG network 
(VGGI16) and the residual network (Resnet50). 
These designs are made up of successive network 
layers that are arranged into "blocks" to regulate 
feature map size. VGG blocks are made up of 3x3 
convolutional layers followed by 2x2 max pooling, 
whereas Resnet blocks are made up of 
convolutional layers with various filter sizes 
connected by shortcut connections. The softmax 
activation function is initially applied to the 


heatmap output of the entire image classifier. 
ei 


j 

fs = Ha for = 1,.-..0 (1) 

The rectified linear unit (ReLU) activation 
is used to overcome gradient flow issues. 

AZ)j = max(0, J) for /= 1,....,c (2) 

By ensuring proper gradient flow during 
training, these equations optimize the learning 
process. The suggested methodology unifies entire 
picture classification and patch recognition into a 
single procedure, improving efficiency and 
facilitating transfer learning. This strategy has 
promising potential for accurate breast cancer 
detection using biomedical mammography images 
when paired with advanced CNN architectures and 
strategic activation functions. 

Figure 1 shows exemplar CNN. 
Convolutional Neural Networks (CNNs) are deep 
learning models used for image classification and 
computer vision tasks. They consist of several 
layers, including the input layer, convolutional 
layer, pooling layer, and fully connected layer. The 
input layer receives raw image data, while the 
convolutional layer extracts features using filters. 
The pooling layer reduces spatial dimensions while 
retaining important information. The fully 
connected layer makes the final classification 
decision by connecting neurons from previous 
layers and forming a fully connected graph. These 
layers perform computations using weights and 
biases. 

Figure 2 shows the general schema for 
image classification based on transfer learning 
which is a deep learning technique that applies 
knowledge from one task to another. It involves 
using a pre-trained CNN model as a feature 
extractor for a new task. The process involves 
selecting a pre-trained model, removing fully 
connected layers, freezing the convolutional layers, 
adding new layers, training the model, fine-tuning, 
evaluating and adjusting the model. 


5. IMPLEMENTATION & RESULTS 

To train an entire image classifier, used a 
three-stage training strategy in which parameter 
learning is frozen for all but the final layer and 
gradually unfrozen from the top to the bottom 
layers while also slowing learning rate. 

In order to train the newly added top 
layers and subsequently train all layers with a 
slower learning rate, a 2-stage training method was 
used for a whole image classifier that was 
converted from a patch classifier. 

In contrast to the Resnet-based image 
classifiers, which had already converged, 
discovered that the VGG-based image classifiers 
showed signs of continuing to improve towards the 
end of the 50 epochs and added 200 more epochs to 
the training of the VGG-based image classifiers. 

On the CBIS-DDSM test set, the 
precision of the categorization of picture patches 
into five classes using ResnetSO and VGG16 was 
evaluated. Pre-trained networks were used for the 
remainder of the study because pre-training can 
significantly improve network convergence and 
performance. All five classes were predicted into 
the appropriate categories with the highest 
probability using the Resnet50 and VGG16 patch 
classifiers. 

Figure 3 shows how to generate region of 
interest (ROD). Creating ROIs from mammogram 
images usually entails identifying possible locations 
that might have breast abnormalities using image 
processing techniques. 

Figure 4 shows radiopaque artifacts 
suppression. Pre-processing is a technique used to 
enhance the homogeneity of mammogram images 
by removing noise and radiopaque artifacts. 


The Resnet50 patch classifiers were 
employed to assess the patch classifiers. The top 
layers of the entire image classifier were 


constructed using the Resnet50 patch classifiers 
because their design surpasses the GPU memory 
limit. The S10 set contains more details about the 
ROIs and surrounding regions than the S1 dataset, 
as evidenced by the fact that the S10 set's patch 
classification accuracy was higher than the S1 set's 
patch classification accuracy. Only patch classifiers 
trained on the S10 dataset were utilised for the 
remainder of the investigation. The outcomes 
demonstrated a low correlation between the 
performance of the entire image classifiers and the 
depths of the Resnet blocks. Networks based on 
VGG performed similarly to those based on Resnet. 
By layering the most effective VGG and Resnet top 
layers on top of one another, produced two hybrid 
networks. The mean AUCs for these two hybrid 
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networks were 0.87 and 0.85, respectively. Note 
that AUC is a measure of a machine learning 
model's performance, indicating its ability to 
distinguish between different data points. 

By averaging the scores of the four 
photographs, we improved the prediction of four 
images. An ensemble model was created from the 
top four models, and it had an AUC of 0.92, a 
specificity of 80.1%, and a sensitivity of 87.1%. 
Using the Resnet-VGG model, we produced 
saliency maps, calculating only positive gradients 
for positive activations using the guided back- 
propagation method. 

The saliency map in an image indicates the 
most prominent and focused pixel, with brightness 
directly proportional to the image's saliency. The 
saliency map of a true positive image demonstrates 
that the cancerous region that the image classifier 
used to make its determination was accurately 
detected. Because each view has the potential to 
contain particular data, combining the CC and 
MLO perspectives for prediction may improve 
performance. When available, using two views 
considerably raised the AUCs in compared to using 
just one view for each of the top four models we 
tested. To enhance the Resnet50 and VGG16 patch 
classifiers' performance, we applied a heatmap, 
max-pooling, and two FC layers. Even with the best 
mean AUC of 0.74, all convolutional models did 
not perform as well as this one. A heatmap was 
added to a Resnet-based whole image classifier, 
however the AUC was noticeably worse than it was 
for the heatmap-free version of the classifier. This 
suggested that deleting the heatmap would be 
advantageous for all image classification networks 
as a whole. 

The AUC for this strategy was 0.73. An 
open database with more recent FFDM images is 
used to perform transfer learning for full image 
categorization on INbreast. 400 mammograms, 
including CC and MLO images, were taken for 115 
individuals, according to the INbreast database. 

Figure 5 shows AUCs graphs comparing 
with different strategies. 

The four top-performing models were 
directly improved upon on the INbreast training set, 
and their performance on the test set was assessed 
by computing per-image AUCs. The AUC was 
increased by the ensemble model to 0.98 with 
86.7% sensitivity and 96.1% specificity. 

Here looked for the bare minimum of 
information needed to optimize a whole image 
classifier's performance to a satisfactory level. 
Discovered that 20 cases were enough, and that 
considerably less data may be needed to acclimatize 


to varying intensity profiles present in various 
mammography datasets. 
5. CONCLUSION 


Deep _learning-based breast cancer 
diagnosis project using biomedical mammography 
pictures is a game-changing development in the 
field of medical diagnostics. This study has shown 
significant potential for early breast cancer 
identification, which could ultimately improve 
patient outcomes and maybe save lives. It does this 
by harnessing the power of convolutional neural 
networks (CNNs) and _ the processing of 
mammography images. The deep learning model 
performed above and beyond expectations, beating 
conventional approaches and _ showcasing its 
aptitude to precisely identify cancerous lesions in 
mammograms. The CNN-based model has 
demonstrated its effectiveness in differentiating 
between benign and malignant cases, minimizing 
the incidence of false negatives and false positives 
in breast cancer screening. It does this by 
automatically extracting complicated patterns and 
subtle elements from the images. This deep 
learning based strategy has a variety of benefits. It 
provides a non-invasive, affordable, and quick 
approach for finding breast cancer, enabling more 
women to get routine screenings and promoting 
early diagnoses. This thus creates the opportunity 
for therapy to start earlier, when there is a far larger 
likelihood of successful intervention and recovery. 
The model is more robust and generalizable as a 
result of the project's emphasis on data 
augmentation and transfer learning. The deep 
learning system adapts well to changing datasets by 
utilizing the knowledge learned from pretrained 
models, assuring its usefulness in various clinical 
contexts and patient groups. However, there is still 
room for advancement and additional study. Multi- 
modal integration, including different imaging 
modalities like ultrasound and MRI, could be used 
in future research to provide a more thorough and 
in-depth investigation of breast tissue. Deep 
learning models must be deployed responsibly and 
ethically in clinical practice, which will require 
continual efforts to address data privacy, ethical 
issues, and potential biases. 

Deep learning for breast cancer diagnosis 
will advance thanks to cooperation between AI 
researchers, medical experts, and stakeholders. To 
verify the model's effectiveness and safety, large 
scale clinical validation studies incorporating a 
variety of patient groups and actual clinical settings 
will be necessary. This will hasten the model's 
adoption into routine breast cancer screening 
programs. In the early identification and 
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management of breast cancer, the marriage of 
cutting-edge technology with medical know-how 
holds enormous promise for bettering healthcare 
outcomes and enhancing the potential of Al-driven 
solutions to improve healthcare. 

6. FUTURE WORK 


Future breakthroughs in the field of breast 
cancer diagnosis using biomedical mammography 
images and deep learning are highly anticipated. 
The integration of multi-modal data, integrating 
mammography pictures with additional imaging 
modalities like ultrasound and MRI, is one 
direction for future research. Such a multimodal 
strategy may give a more thorough view of breast 
tissue and increase the precision of cancer 
detection, particularly in complex cases. 

The creation of sophisticated data 
augmentation methods designed especially for 
mammography pictures is another encouraging 
direction. Limited labelled data can be overcome by 
producing a variety of synthetic data and domain- 
specific augmentations, improving the model's 
capacity to generalize to new datasets and other 
patient demographics. 

Additionally, investigating —_ poorly 
supervised learning approaches offers an attractive 
opportunity to lessen the workload associated with 
manual lesion labelling. Weakly supervised 
approaches may enable access to larger datasets and 
speed up model construction by working with 
image-level labels rather than in-depth annotations. 
The assessment of uncertainty is yet another 
significant subject for future study. The creation of 
techniques to measure the model's uncertainty in its 
forecasts may result in more transparent and 
reliable clinical decision-making. In circumstances 
where additional expert assessment or diagnostic 
testing could be necessary, this capacity is 
extremely beneficial. 

The interpretability of AI systems must 
be improved if deep learning models are to be 
successfully integrated into clinical practice. 
Radiologists can have more confidence in the 
model's decision-making process if explainable AI 
techniques are the subject of ongoing study. Real- 
world extensive clinical validation studies with a 
range of patient demographics and healthcare 
settings. The effectiveness, reliability, and safety of 
the model will be evaluated in these studies, 
opening the door for widespread clinical use. 

Future work will continue to be 
critically dependent on ethical issues. In order to 
safeguard patient rights and provide equal access to 
breast cancer screening technologies, it is crucial to 
address data privacy, fairness, and potential biases 


in the model. Exploring the model's performance in 
identifying breast cancer in high-risk populations, 
such as women with a family history of the disease 
or genetic abnormalities, can also result in the 
development of specialized screening methods and 
individualized healthcare. 

Last but not least, there is considerable 
potential in researching the application of deep 
learning models for following breast cancer patients 
throughout time. These models might be used to 
monitor the development of a disease, gauge how 
well a medication is working, and develop 
individualized treatment regimens over time. There 
is great potential in the work that will be done in 
the future to detect breast cancer using biomedical 
mammography pictures and deep learning. 
Researchers can progress the discipline to enhance 
patient outcomes and early detection rates by 
investigating multi-modal integration, enhanced 
data augmentation, uncertainty estimation, and 
ethical considerations. In order to accelerate these 
developments and ultimately have a significant 
impact on the fight against breast cancer, 
collaboration between AI scientists, medical 
professionals, and stakeholders will be crucial. 
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Figure 3: Generate candidate ROIs 
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Figure 4: Radiopaque artifacts suppression 
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Figure 5: AUCs graphs comparing with different strategies 
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